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PREFACE 


This  Memorandum  discusses  some  of  the  problems  involved 
in  the  formulation  of  a  mathematical  theory  for  system 
identification. 

Professor  Bellman,  of  the  University  of  Southern 
California,  is  a  consultant  to  The  RAND  Corporation. 
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SUMMARY 

Let  us  consider  a  complex  system  consisting  of  a 
set  of  interacting  subsystems.  Let  x^(t)  represent 
the  state  vector  for  the  i— th  subsystem,  and  suppose  that 
the  behavior  over  time  of  these  vectors  is  determined  by 
a  set  of  coupled  functional  equations 

dx. 

dt“  "  8i(xi^2,...,x5,ai),  i  =  1,2, 3, 4, 5, 

where  the  a^  are  parameters  determining  both  the 
structure  of  the  subsystems  and  the  linkage  between  these 
subsystems. 

Much  of  classical  analysis  is  devoted  to  the 
qualitative  and  quantitative  analysis  of  the  x^(t)  as 
functions  of  t  and  of  the  structural  parameters. 
Important  as  this  effort  is,  it  represents  only  a  part  of 
the  principal  objective,  which  is  that  of  the  identifica¬ 
tion  of  physical  systems.  By  the  "identification  of 
systems,"  we  mean  the  task  of  determining  the  structural 
parameters  on  the  basis  of  observations  over  time  and 
position  of  the  inputs  and  outputs.  This  is  an  essential 
part  of  the  validations  of  hypotheses  and  theories. 


ON  THE  CONSTRUCTION  OF  A  MATHEMATICAL  THEORY  OF  THE 
IDENTIFICATION  OF  SYSTEMS 


1.  INTRODUCTION 

Let  us  consider  a  complex  system  consisting  of  a 
set  of  interacting  subsystems: 


Let  x^(t)  represent  the  state  vector  for  the  i— th 
subsystem,  and  suppose  that  the  behavior  over  time  of 
these  vectors  is  determined  by  a  set  of  coupled 
functional  equations 

dx. 

(1*1)  dt-  =  ®i ai^  *  **  1*2,3, 4,5, 

where  the  a^  are  parameters  determining  both  the 
structure  of  the  subsystems  and  the  linkage  between  these 
subsystems. 

Mich  of  classical  analysis  is  devoted  to  the 
qualitative  and  quantitative  analysis  of  the  x^(t)  as 
functions  of  t  and  of  the  structural  parameters. 
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Important  as  this  effort  is,  it  represents  only  a  part  of 
the  principal  objective,  which  is  that  of  the 
identification  of  physical  systems.  By  the  "identification 
of  systems,"  we  mean  the  task  of  determining  the 
structural  parameters  on  the  basis  of  observations  over 
time  and  position  of  the  inputs  and  outputs.  This  is  an 
essential  part  of  the  validations  of  hypotheses  and 
theories. 

There  are  many  levels  to  this  identification  process. 
Let  us  cite  some  of  the  problems  encountered  at  the 
lowest  level: 


1.  Given  a  set  of  observations,  [  (x^(t. )  ,c^  )  } , 
i  «  1,2,...,M,  j  “  1,2,...,N,  to  what  extent  are  the 


a^..  determined,  and  how  do  we  construct  algorithms  for 
their  determination? 


2.  Where  should  the  inputs  to  the  system  be  applied 
and  the  outputs  measured?  How  does  one  do  this  so  as  to 
minimize  the  disturbance  both  to  behavior  of  the  system 
and  to  the  calculated  values  caused  by  the  testing 
process? 

3.  At  what  times  should  the  observations  be  made? 


4*  What  accuracy  is  required  in  measurement  to 
obtain  a  desired  accuracy  for  the  a^? 

5.  What  kinds  of  measuring  instruments  should  be 
used  and  what  types  of  stimuli  applied?  Analytically, 
this  may  be  considered  to  be  a  determination  of  the 


functions 


in  the  observations  cp^Cx^Ct.)) 


6.  What  data  are  desired?  Analytically,  this  may 
be  considered  to  be  a  determination  of  functions  ^,(a^). 

These  questions,  difficult  as  they  are,  are  only 
preliminary  to  the  study  of  enveloping  problems  of 
identification  and  control.  These,  in  turn,  introduce 
interesting  and  recondite  stochastic  and  adaptive 
aspects;  see  [1]. 

All  of  this  in  its  turn  is  prologomena  to  the 
construction,  of  a  meaningful  theory  of  information,  and, 
finally,  an  embracing  theory  of  instrumentation  and 
experimentation . 

In  what  follows,  we  wish  to  describe  briefly  some 
mathematical  techniques  we  have  employed  to  treat  the 
first  of  the  problems  posed  above,  and  to  indicate  some 
of  the  applications  we  have  made.  As  the  reader  will 
see,  the  mathematical  level  is  not  very  high.  What  we 
have  been  particularly  concerned  with  is  an  examination 
of  how  far  one  can  penetrate  into  this  domain  with 
relatively  simple  analytic  techniques  tailored  to  the 
use  of  a  contemporary  digital  computer.  Essentially,  we 
have  attempted  to  construct  a  handbook  of  soluble 
problems  [2],  and  also  to  carry  out  systematic 
mathematical  experimentation  in  this  most  important 
scientific  area.  What  is  remarkable  is  that  many 
significant  questions  can  be  resolved  using  the  ideas 
sketched  below. 
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2.  A  FUNDAMENTAL  PROBLEM 

Consider  a  vector  differential  equation 

(2.1)  x  -  g(x,a),  x(0)  -  c, 

where  x  is  N— dimensional  and  a  is  M-dimensional. 
Suppose  that  we  are  given  a  set  of  observations 
{ (x(t^)  ,b^)  } ,  i  »  1,2,...,R,  and  asked  to  determine  a 
and  c  on  this  basis. 

There  are  several  distinct  classes  of  problems  that 
may  be  distinguished  here.  The  first  is  that  where  we 
suppose  that  x(t)  does  indeed  satisfy  an  equation  such 
as  (2.1)  and  that  a  and  c  exist.  The  second  is  a 
design  problem  where  a  and  c  are  to  be  determined  so 
that  (x(t^),b^)  «  d^,  i  ■  1,2,...,N.  The  third  is 
where  we  know  in  advance  that  x(t)  does  not  satisfy  an 
equation  of  as  simple  a  type  as  (2.1),  but  where  we  want 
to  find  the  best  approximate  equation  of  this  structure. 
This  gets  us  into  the  area  of  differential  approximation 
[2,3,4]. 

Let  us  write  d^  for  the  actual  observation  at  time 
i  and  determine  a  and  c  by  the  condition  that  the 
quantity 


E  ((x(t.),b.)  -  d.) 
i-1  ill 


(2.2) 


be  minimized.  This  analytic  problem,  which  can  be 
handled  in  various  ways  as  we  shall  indicate  below,  is 
common  to  all  three  classes  of  questions. 

3.  SEARCH  TECHNIQUES 

As  an  indication  of  a  simple  and  direct  approach, 
let  us  consider  the  problem  of  determining  X  in  the 
Van  der  Pol  equation 

(3.1)  u"  +  X(u2  -  l)u *  +  u  -  0, 

on  the  basis  of  observations,  u(t^)  *  d^,  i  «  1,2,...,R. 
Letting  u(t,X)  denote  the  unique  periodic  solution  of 

(3.1) ,  X  >  0,  we  wish  to  minimize 

R  o 

(3.2)  f (X)  -  Z  (u(t.,X)  -  d.)Z 

i-1  1  1 

over  X  >  0. 

Choosing  a  value  of  X,  the  determination  of  the 
set  of  values  {u(t^,X)}  for  R  of  the  order  of 
magnitude  of  10,  100,  1000,  consumes  a  few  seconds  of 
computer  time — at  most.  Consequently,  we  can  begin  by 
evaluating  f (0.1) ,f (0.2), . . . ,f (l),f  (2), . . .,f (10),  and 
thereby  determine  an  interval  within  which  the  correct 
value  of  X  lies.  Having  obtained  this  interval,  we  can 
subdivide  it  into  ten  parts,  and  so  on.  In  this  way,  by 
means  of  a  few  minutes  of  computer  time,  we  can  determine 
X  accurately  to  several  significant  figures. 


In  more  complex,  multidimensional  minimization 
problems,  it  will  pay  to  use  sophisticated  search  methods 
of  the  Kiefer-Johnson  variety  [5]. 

The  point  we  wish  to  emphasize  is  that  the  capabilities 
of  modem  computers  frequently  permit  us  to  approach 
problems  of  this  nature  in  a  simple  and  direct  fashion. 

4 .  QUASILINEARIZATION 

A  systematic  approach  to  the  multidimensional 
minimization  problem  described  in  (2.2)  is  supplied  by 
the  theory  of  quasilinearization  [2],  which  in  a  number 
of  cases  yields  techniques  which  can  be  considered  to  be 
extensions  of  the  Newton-Raphson-Kantorovich  approximation 
method  [6]. 

The  basic  idea  in  this  case  is  quite  simple.  Let 
a^,  c<°>  be  initial  guesses  as  to  the  values  of  a 
and  c  and  let  be  computed  as  the  solution  of 


(4.1) 


(0)  .  g(x(0) ,„«»), 


x(0)(0) 


(0) 


-  These  values  are  obtained  either  on  the  basis  of 
experience  and  intuition,  through  the  use  of  simpler 
models,  or  by  way  of  search  techniques  of  the  type 
described  above,  or  a  combination  of  all  three.  The 
next  approximations  are  obtained  in  the  following  fashion. 
Expand  g(x,a)  around  x^,  a^ ,  keeping  zero  and 
first-order  terms,  and  let  x^  be  determined  as  the 
solution  of 
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(4.2)  x^  *=  g(x^°\a^0^)  +  J^(x^  -  x^) 


+J0(a<l>-a<°>), 


x^(0)  =  c^.  Here  and  J2  are  Jacobian  matrices. 

The  vectors  a^  and  c^  are  now  obtained  by  way  of 
the  minimization  of  the  quadratic  form 


(4.3) 


£ 

E  «x(1)(t.),b.)  -  d.)2, 
i=l  111 


We  have  thus  reduced  the  original  optimization  problem 
to  a  succession  of  operations  involving  the  numerical 
solution  of  linear  differential  equations  and  the 
numerical  solution  of  linear  algebraic  equations.  These 
operations  can  be  carried  out  accurately  and  quickly  for 
systems  of  quite  high  dimension.  A  detailed  discussion 
of  this  method,  with  numerous  applications,  will  be 
found  in  [ 2 ] . 

5.  USE  OF  TRANSFORM  TECHNIQUES 

Functional  equations  of  more  complex  type,  such  as 
the  heat  equation. 


(5.1)  k(x)ut  -  u^. 


where  k(x)  ■  k(x,a),  and  a  is  an  unknown  parameter, 
or  a  differential— difference  equation 


(5.2) 
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u'  (t)  =  au(t  -  t)  , 

where  both  a  and  the  time-lag  t  are  unknown,  may  be 
handled  directly  by  means  of  the  techniques  described 
above,  or  by  use  of  Laplace  transform  techniques.  In 
this  latter  case,  we  use  the  foregoing  method  as  applied 
to  the  resulting  differential  equations  in  the  transform 
space  [7],  [8]. 

6.  BASIC  UNCERTAINTIES  OF  ON-LINE  IDENTIFICATION  AND  CONTROL 
Let  us  consider  briefly  some  measurement  and  data 
processing  aspects  of  large  systems.  Systems  with 
stochastic  effects  exhibit  similar  properties.  Suppose 
we  are  attempting  to  implement  a  policy  at  time  t  which 
depends  upon  a  knowledge  of  the  components  of  the  state 
vector  x(t).  If  the  dimension  of  x  is  large,  we  must 
accept  the  fact  that  an  appreciable  time  will  be  involved 
in  the  measurement  of  all  of  the  components  and  the 
transmission  of  the  required  information  to  the  decision¬ 
maker.  During  this  time  the  system  is  uncontrolled, 
which  means  that  the  system  is  not  operating  in  its 
optimal  fashion.  It  follows  then  that  there  are  two 
extremes:  Act  instantaneously  on  the  basis  of  a  minimum 
of  information;  act  on  the  basis  of  a  maximum  information 
at  a  later  time.  Both  extreme  policies  will  introduce 
errors  and  associated  costs.  Various  simple  models  of 
control  and  decision  processes  of  this  type  will  yield 
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analytic  estimates  which  are  similar  in  structure  to  the 
classical  uncertainty  principles  of  quantum  mechanics. 

The  multistage  nature  of  the  identification  and 
control  process  enables  us  to  diminish  considerably  the 
effect  described  above.  One  might  expect  that  similar 
multistage  measurement  techniques  applied  in  physics 
would  reduce  the  uncertainties  in  quantum  mechanics.  In 
any  case,  there  are  a  number  of  fascinating  questions  in 
these  areas  associated  with  the  allocation  of  time, 
energy,  and  other  resources  to  most  effective  identifica¬ 
tion,  which  we  shall  discuss  at  another  time. 

7.  DECOMPOSITION  INTO  SUBSYSTEMS 

In  many  cases  of  interest,  the  problem  is  that  of 
determining  the  structural  properties  of  a  system  in 
position  or  time,  or  both.  Thus,  for  example,  both  in 
geophysics  and  in  astrophysics,  we  wish  to  determine  the 
stratification  of  a  medium  on  the  basis  of  observations 
made  at  one  interface.  The  theory  of  invariant  imbedding 
can  be  applied  in  these  cases;  see  [2],  [9]. 

Inputs  Observations 
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In  the  study  of  metabolic  processes,  and  in  certain 
areas  of  neurophysiology,  we  encounter  a  stratification 
in  time.  We  know  that  the  time  history  of  the  system  is 
governed  by  equations  of  the  following  form, 

(7.1)  =  s(x,a^),  t^  ^  t  i-  ~  0,1,2,..., 

and  the  problem  is  to  determine  the  structural  vectors 
a^  and  the  transition  times  t^  on  the  basis  of  a  set 
of  observations  {  (x(t.  )  ,b^ )  } .  A  discussion  of  how 
dynamic  programming  may  be  applied  to  problems  of  this 
and  the  foregoing  type  will  be  found  in  [2,10,11]. 
Discussions  of  the  application  of  a  combination  of 
dynamic  programming  and  quasilinearization  will  be  found 
in  [12,13,14,15]. 

8.  PATTERN  RECOGNITION 

It  is  clear  that  the  problem  of  decomposition  into 
subsystems  is  a  special  case  of  the  general  problem  of 
pattern  recognition.  In  turn,  pattern  recognition  can 
be  viewed  as  a  dynamic  programming  process  and  particular 
problems  can  be  handled  by  means  of  the  functional 
equation  technique;  see  [16,17,18,19]. 

9.  CONCLUDING  REMARKS 

We  hope  that  the  foregoing  has  made  it  clear  that 
the  0e  .  1  problem  of  the  identification  of  systems  is 
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one  of  the  most  Important  and  challenging  in  modern 
science.  Many  ew  types  of  mathematical  problems  arise. 
Some  we  have  s  d  explicitly,  some  we  have  hinted  at, 
but  most  have  been  passed  ovu  in  silence  for  want  of 
precise  statement  or  formulation.  With  practically  no 
past,  and  a  glorious  future,  this  is  an  ideal  field  for 
the  young  mathematician. 
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